In addition, we explore Processing (NLP) have introduced an immense methods to improve the robustness of LMs to improvement in many aspects, including code-mixed text. Using our IndoRobusta-Shot, standardized benchmarks (Wilie et al., 2020; we perform adversarial training to improve the Cahyawijaya et al., 2021; Koto et al., 2020; Winata code-mixed robustness of LMs. We explore three et al., 2022), large pre-trained language model kinds of tuning strategies: 1) code-mix only, 2) (LM) (Wilie et al., 2020; Cahyawijaya et al., 2021; two-steps, and 3) joint training, and empirically Koto et al., 2020), and resource expansion covering search for the best strategy to improve the model local Indonesian languages (Tri Apriani, 2016; robustness on code-mixed data.

First, it came the language model. The intuition was easy: the next word in a sequence of words can be modeled with a probability distribution and is heavily dependent on the previous words. Words are part of a vocabulary, a limited corpus (170,000 tokens in the English vocabulary). Each word has a limited amount of meanings. Which is a predictable structure.

ANJA KASPERSEN: Today I am very pleased to be joined by Pascale Fung. Pascale is a;rofessor in the Department of Electronic and Computer Engineering and Department of Computer Science and Engineering at The Hong Kong University of Science and Technology. She is known globally for her pioneering work on conversational artificial intelligence (AI), computational linguistics, and was one of the earliest proponents of statistical and machine-learning approaches for natural language processing (NLP). She is now leading groundbreaking research on how to build intelligent systems that can understand and empathize with humans. I have really been looking forward to this conversation with you. Your professional accolades are many, most of which we will touch on during our conversation. However, for our listeners to get to know you a bit better, I would like us to go back to your upbringing during what I understand to be a very tenuous political period in China. I was born, spent my childhood, ...

Learning to converse using only a few examples is a great challenge in conversational AI. The current best conversational models, which are either good chit-chatters (e.g., BlenderBot) or goal-oriented systems (e.g., MinTL), are language models (LMs) fine-tuned on large conversational datasets. Training these models is expensive, both in terms of computational resources and time, and it is hard to keep them up to date with new conversational skills. A simple yet unexplored solution is prompt-based few-shot learning (Brown et al. 2020) which does not require gradient-based fine-tuning but instead uses a few examples in the LM context as the only source of learning. In this paper, we explore prompt-based few-shot learning in dialogue tasks. We benchmark LMs of different sizes in nine response generation tasks, which include four knowledge-grounded tasks, a task-oriented generations task, three open-chat tasks, and controlled stylistic generation, and five conversational parsing tasks, which include dialogue state tracking, graph path generation, persona information extraction, document retrieval, and internet query generation. The current largest released LM (GPT-J-6B) using prompt-based few-shot learning, and thus requiring no training, achieves competitive performance to fully trained state-of-the-art models. Moreover, we propose a novel prompt-based few-shot classifier, that also does not require any fine-tuning, to select the most appropriate prompt given a dialogue history. Finally, by combining the power of prompt-based few-shot learning and a Skill Selector, we create an end-to-end chatbot named the Few-Shot Bot (FSB), which automatically selects the most appropriate conversational skill, queries different knowledge bases or the internet, and uses the retrieved knowledge to generate a human-like response, all using only few dialogue examples per skill.

Information-seeking dialogue systems, including knowledge identification and response generation, aim to respond to users with fluent, coherent, and informative responses based on users' needs, which. To tackle this challenge, we utilize data augmentation methods and several training techniques with the pre-trained language models to learn a general pattern of the task and thus achieve promising performance. In DialDoc21 competition, our system achieved 74.95 F1 score and 60.74 Exact Match score in subtask 1, and 37.72 SacreBLEU score in subtask 2. Empirical analysis is provided to explain the effectiveness of our approaches.

Considerable progress has been made towards conversational models that generate coherent and fluent responses by training large language models on large dialogue datasets. These models have little or no control of the generated responses and miss two important features: continuous dialogue skills integration and seamlessly leveraging diverse knowledge sources. In this paper, we propose the Adapter-Bot, a dialogue model that uses a fixed backbone conversational model such as DialGPT (Zhang et al., 2019) and triggers on-demand dialogue skills (e.g., emphatic response, weather information, movie recommendation) via different adapters (Houlsby et al., 2019). Each adapter can be trained independently, thus allowing a continual integration of skills without retraining the entire model. Depending on the skills, the model is able to process multiple knowledge types, such as text, tables, and graphs, in a seamless manner. The dialogue skills can be triggered automatically via a dialogue manager, or manually, thus allowing high-level control of the generated responses. At the current stage, we have implemented 12 response styles (e.g., positive, negative etc.), 8 goal-oriented skills (e.g. weather information, movie recommendation, etc.), and personalized and emphatic responses. We evaluate our model using automatic evaluation by comparing it with existing state-of-the-art conversational models, and we have released an interactive system at adapter.bot.ust.hk.

It's widely accepted that artificial intelligence (AI) will have a huge impact on our lives in the coming decades -- but what's its value to the global economy? According to a new report, global GDP will be 14% higher in 2030 as a result of AI -- the equivalent of $15.7 trillion, more than the current output of China and India combined. The report, Sizing the Prize, was launched by PwC in a session at the World Economic Forum's Annual Meeting of the New Champions 2017 in Dalian, China. Improvements to labour productivity will account for over half of all economic gains from AI between now and 2030, while increased consumer demand resulting from product enhancements will account for the rest. Regional gains will be most strongly felt in China, which will receive a 26% boost to GDP in 2030, followed by North America (14.5%).